(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 




(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date (10) International Publication Number 

8 February 2001 (08.02.2001) PCT WO 01/10128 Al 

(51) International Patent Classification 7 : H04N 7/173 DODGE, Christopher; 30 Allen Street, Arlington, MA 

02474 (US). BOISSIERE, Guillaume; Apartment 505, 

(21) International Application Number: PCTAJS00/21214 950 Massachusetts Avenue, Cambridge, MA 02139 (US). 

(22) International Filing Date: 3 August 2000 (03.08.2000) (74) Agent: MILSTEIN, Joseph, B.; Testa, Hurwitz & 

Thibeault, LLP, High Street Tower, 125 High Street, 

(25) Filing Language: English Boston ' MA 021 10 < US )' 

(26) Publication Language: English ( 81 ) Designated States (national): AU, CA, CN, IL, JP. 

(30) Priority Data: (84) Designated States (regional): European patent (AT, BE, 

60/147,029 3 August 1999 (03.08.1999) US CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, 

09/497,587 3 February 2000 (03.02.2000) US NL, PT, SE). 

60/196,069 10 April 2000 (10.04.2000) US 

Published: 

(71) Applicant: VIDEOSHARE, INC. [US/US]; Third Floor, — With international search report. 
907 Massachusetts Avenue, Cambridge, MA 02139 (US). 

For two-letter codes and other abbreviations, refer to the "Guid- 

(72) Inventors: LIWERANT, Gad; Apartment 608, 1008 ance Notes on Codes and Abbreviations" appearing at the begin- 
Massachusetts Avenue, Cambridge, MA 02138 (US). ning of each regular issue of the PCT Gazette. 



(54) Title: INSTANT VIDEO MESSENGER 



"command and 
control" link to IVM 
server 



IVM 



-406 

- "command and 
control" link to IVM 
server 



302-^ 



A 




B 


> 

direct video 



— 304 



stream served to B 



f*^ (57) Abstract: Methods and systems for a sender of a video message to send the message over a computer network from the sender's 
computer directly to a receiver computer used by a recipient, without the intermediation of a server computer. A server computer 
determines whether the receiver computer is online. In response to a request from the sender computer, the server informs the 
- sender computer of the online status of the receiver computer. The sender computer prepares a viedo message, mcluding a video 
ftm ^ segment, and optionally audio, text an other information. The sender computer informs the receiver computer that a video message 
^ is available to be sent. Depending on the response of the receiver computer, the video message is streamed from the sender computer 
to the receiver computer, the video message is sent to a host computer for storage and later streaming to the receiver computer, or 
£^ is not sent. If the receiver computer is not online, the video message is optionally sent to the host computer for storage and later 
^* streaming to the receiver computer. 
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INSTANT VIDEO MESSENGER 

Cross-References to Related Applications 

This application claims priority to and the benefit of U.S. Provisional Patent Application 
Serial Number 60/147,029, filed August 3, 1999, and of U.S. Provisional Patent Application 
Serial Number 60/196,069, filed April 10, 2000, which applications are incorporated herein in 
their entirety by reference. This application is a Continuation-in-Part of U.S. Patent Application 
Serial No. 09/497,587 (hereinafter "U.S.S.N. 09/497,587"), filed February 3, 2000, entitled 
"Method And System For Sharing Video Over A Network", which application is incorporated 
herein in its entirety by reference. 

Field of the Invention 

This invention relates generally to transmitting video files over a network. More 
particularly, the invention relates to transmitting streaming video files over a network without the 
intervention of a server computer in the streaming process. 

Background of the Invention 

Communication of information in the form of digital or computer files, such as 
documents, videos, and the like, by attachment to electronic messages such as electronic mail is 
well known. With this type of transmission, the entire video file must be transmitted and 
received before the receiver can view the video. For large files, the time required to complete 
such transmissions can be longer than the actual playing time of the video. Also, this type of 
transmission typically requires multiple computer programs to perform all of the necessary 
functions, including an e-mail application program to send or receive the video in computer file 
form, and a second program to play or display the video from the received file attachment. While 
these forms of communication represent an advance over the transmission of hard copies of 
documents, videos, and the like, there are still many limitations and problems that are associated 
with such communications. 

Some of the problems that are associated with communications over a computer network 
operated according to the client/server model (and thus including one or more server computers) 
include, but are not limited to, the large expense of building, maintaining, and operating servers 
generally, and in particular the expense of installing and maintaining servers that perform 
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functions related to data storage and hosting; the problems associated with loss of service if the 
server is inoperative, and degradation of service if the server is overloaded; delays, 
inconvenience, and bandwidth capacity issues associated with uploading information from a 
sender computer to the server for retransmission from the server to a recipient computer; and 
difficulties in increasing the scale of the network as the number of client computers increases. 
Because video files in particular tend to be quite large, the problems enumerated above can be 
particularly severe in handling video files in a client/server network. 

Summary of the Invention 
The present invention provides a method and system of transmitting a video that 
mitigates the problems associated with the lengthy transmission time associated with a file 
attached to an e-mail, that eliminates the necessity to have multiple software packages to carry 
out the transmission, and that eliminates the necessity to invoke the services of a hosting 
computer for storing the video. In some embodiments, the method and system can be transparent 
to the user. In accordance with the present invention, full motion video can be automatically 
prepared and transmitted by a sender from a sender computer to a viewer using a receiver 
computer, when the two computers are simultaneously connected to a computer network, 
including such networks as a global computer network, for example, the Internet or the World 
Wide Web ("Web"). 

The invention is described in terms of an exemplary embodiment called the Instant Video 
Messenger (IVM). IVM is a rapid, asynchronous messaging system/software that supports 
symmetric, peer-to-peer communication of digital information (e.g., files containing video data 
such as files in the jpg, .avi, .mpeg, QuickTime, .wav, Windows Media Format, Real Media, and 
other formats) on a computer network, such as the Internet. Each user computer equipped with 
the IVM software is capable of sending and receiving streaming and non-streaming audio, 
streaming and non-streaming video, text, attachments, and meta-data. The network structure of 
IVM reduces the burden on a central server (which traditionally has been used to host and stream 
video), as IVM is a peer-to-peer application and thus does not require a hosting facility. A server 
can be used in an IVM system to perform functions that include, but are not limited to, 
monitoring which IVM users are on-line, providing communications driven by an entity that has 
permission to use the server (for example, an advertiser who has contracted for services), and 
hosting messages for users who have contracted for that service. IVM supports an Instant Video 
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Messenger Channel capability that allows for community building as well as affiliate 
opportunities. The IVM distribution system allows for "self-replication." 

In one aspect, the invention features a method of sending a streaming video message over 
a network. The method includes receiving at a server computer an indication that the operator of 
a sender computer intends to send a streaming video message to a receiver computer over a 
network, determining whether the receiver computer is online, and communicating from the 
server computer to the sender computer whether the receiver computer is online, wherein an 
indication that the receiver computer is online signals to the sender computer that streaming the 
video message directly from the sender computer to the receiver computer over the network is 
appropriate. 

In one embodiment, the invention can include a supervisory computer communication 
module operable on the server computer that performs the step of receiving at the server 
computer an indication that the operator of a sender computer intends to send a streaming video 
message to a receiver computer. The supervisory computer communication module can also 
perform the step of determining whether the receiver computer is online, and can perform the 
step of communicating from the server computer to the sender computer whether the receiver 
computer is online. 

In one aspect, the invention features a method of sending a streaming video message over 
a network. The method includes sending to a server computer an indication that the operator of a 
sender computer intends to send a streaming video message to a receiver computer over a 
network, receiving at the sender computer an indication of whether the receiver computer is 
online, and in response to an indication that the receiver computer is online, streaming the video 
message from the sender computer to the receiver computer over the network independent of the 
operation of the server computer. 

In one embodiment, the step of streaming the video message from the sender computer to 
the receiver computer includes providing on the sender computer a computer communication 
module operable on each of the sender computer and the receiver computer, and streaming the 
video message from the communication module of the sender computer to the receiver computer 
over the network independent of the operation of the server computer. The step of providing a 
computer communication module operable on each of the sender computer and the receiver 
computer can include providing individual copies of the same communication module to at least 
one of the sender computer and the receiver computer, each copy operable on each of the sender 
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computer and the receiver computer. The method can include providing communication 
modules that are symmetric in operation. The method can include providing communication 
modules that are modular, and can include providing communication modules that can determine 
the features present in another communication module operating on another computer. 

In another embodiment, the step of streaming the video message from the sender 
computer to the receiver computer over the network independent of the operation of the server 
computer can include performing the streaming operation using only communication modules 
present on the sender computer and the receiver computer. 

In a further embodiment, the method can additionally include the step of, in response to 
an indication that the receiver computer is not online, optionally sending the video message from 
the sender computer to a host computer, for storage and later transmission to the receiver 
computer, independent of the operation of the server computer. 

In still another embodiment, the video message can include video information. The video 
message can additionally include information selected from audio information, a thumbnail 
image, a text identifying a subject, a message body text, a document attachment, a slide and 
meta-data. 

In one aspect, the invention features a computer program recorded on a machine-readable 
medium. The computer program is adapted to transmit and receive streaming video messages 
over a network independent of a server computer. When the computer program operates on a 
sender computer, it performs the steps of requesting from a server computer the status of a 
receiver computer, the receiver computer being the intended recipient of a video message, 
receiving an indication of the online status of the receiver ComputerLand in response to an 
indication that the receiver computer is online, streaming the video message from the computer 
program of the sender computer to the computer program of the receiver computer independent 
of the operation of the server computer. 

In one embodiment, when it is operating on a receiver computer, the computer program is 
further capable of determining that a video message is being sent by a sender computer. The 
program is additionally capable of displaying to a viewer of the receiver computer information 
about the video message, the information comprising at least one of an identifier of the sender of 
the message, a subject of the message, and a title of the message. 

In another embodiment, when it is operating on a receiver computer, the computer 
program is capable of receiving the streaming video message, and displaying the streaming video 
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message to a viewer of the receiver computer. The step of receiving the streaming video 
message can include receiving the streaming video message, and presenting to a viewer of the 
receiver computer an opportunity to select one of accepting the streaming video message, and 
rejecting the streaming video message. The step of receiving the streaming video message can 
include accepting a response of the viewer, and processing the message in accordance with the 
response of the viewer. The step of accepting the streaming video message can include at least 
one of accepting the streaming video message for immediate display, and accepting the streaming 
video message for storage in a machine-readable memory, and for display at a later time. 

In still another embodiment, when it is operating on a receiver computer, the computer 
program is capable of receiving the streaming video message is accomplished without the 
necessity of an action by a viewer of the computer. In this embodiment, the program can 
accomplish displaying the streaming video message without the necessity of an action by a 
viewer of the computer. The computer program can perform displaying the video message by 
invoking the operation of a video display module. 

In another embodiment, the computer program can, in response to an indication that the 
receiver computer is not online, optionally send the video message from the communication 
module of the sender computer to a host computer, for storage and later transmission to the 
communication module of the receiver computer^ independent of the operation of the server 
computer. 

In yet another embodiment, the video message can include video information. The video 
message can further include information selected from audio information, a thumbnail image, a 
text identifying a subject, a message body text, a document attachment, a slide and meta-data. 

In one aspect, the invention features a computer program recorded on a machine-readable 
medium. The computer program is adapted to operate a server computer and to provide server 
functionality including database connectivity and communication protocols. When operating on 
a server computer, the computer program performs the steps of determining the online status of 
the receiver computer, and providing an indication of the status to the sender computer, such that 
in response to an indication that the receiver computer is online, streaming the video message 
from the computer program of the sender computer to the computer program of the receiver 
computer is accomplished independent of the operation of the server computer. 

In one embodiment, when in operation, the computer program is capable of accepting a 
request from a sender computer about the status of a receiver computer, the receiver computer 
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being the intended recipient of a video message. When in operation, the computer program is 
capable of providing communication protocols for exchanging information between 
interconnected computers. When in operation, the computer program is capable of providing a 
database in which to store information, including information about sender and receiver 
5 computers authorized to communicate with the server computer. 

The foregoing and other objects, aspects, features, and advantages of the invention will 
become more apparent from the following description and from the claims. 
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Brief Description of the Drawings 
The objects and features of the invention can be better understood with reference to the 
drawings described below, and the claims. The drawings are not necessarily to scale, emphasis 
instead generally being placed upon illustrating the principles of the invention. In the drawings, 
5 like numerals are used to indicate like parts throughout the various views. 

FIG. 1 shows the relationship between the time that elapses between the sending and the 
receiving of a message ("immediacy") and data delivery solutions of some technology such as 
that described in copending U.S. S.N. 09/497,587. 

FIG. 2A depicts the interactions in a system including a sender computer, a server 
10 computer, and a receiver, such as the system described in copending U.S. S.N. 09/497,587. 

FIG. 2B presents in overview a flowchart showing the steps involved in the interaction 
depicted in FIG. 2A. 

FIG. 3 shows the relationship between immediacy and data delivery solutions of FIG. 1 
and additionally including an embodiment of a method and system according to the invention. 
15 FIG. 4A depicts an embodiment of the interactions between and among a sender 

computer, a server computer, and a receiver computer in a network according to the invention. 

FIG. 4B presents in overview a flowchart showing the steps involved in the interaction 
depicted in FIG. 4A. 

FIG. 5 depicts another embodiment of the interactions between and among a sender 
20 computer, a server computer, and a receiver computer in a network according to the invention. 

FIG. 6 depicts an embodiment of the interactions between and among a sender computer, 
a server computer, and a receiver computer in a network having Instant Video Messenger 
Channels according to the invention. 

FIG. 7 depicts an embodiment of the interactions between and among a sender computer, 
25 a server computer, and a receiver computer in a network having Instant Video Messenger 
Channels and the capacity to store message data according to the invention. 

FIG. 8 is a flow diagram that depicts schematically an alternative embodiment of the 
messaging systems and methods, according to the principles of the invention. 

FIG. 9 is a schematic embodiment of a process and system according to the copending 
30 U.S.S.N. 09/497,587. 

FIG. 10 is an embodiment of a system according to the copending U.S.S.N. 09/497,587, 
including the interactions and interrelationships within the system. 
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FIG. 1 1 is a functional block and flow diagram of an embodiment of the copending 
U.S.S.N. 09/497,587. 

FIG. 12 is a login screen on a user's computer, in one embodiment of the invention. 

FIG. 13 is a record/playback screen as seen by the user, in accordance with an 
embodiment of the invention. 

FIG. 14A is a flow diagram of an embodiment of the invention in which software 
automates a number of steps in connection with the uploading of a video segment. 

FIG. 14B is a flow diagram of another embodiment of the invention in which software 
automates a number of steps in connection with the uploading of a video segment. 

FIG. 14C is a flow diagram of an embodiment of the invention in which software 
automates a number of steps in connection with the formatting of a video segment. 

FIG. 14D shows the relationship of some of the files created in the flow diagram of FIG. 

14C. 

FIG. 14E is a flow diagram of a method by which an optimally formatted video segment 
is sent to a user according to the invention. 

FIG. 15 is a screen as seen by the user, the screen indicating that file processing is 
occurring. 

FIG. 16 is a video playback screen seen by the user. 
FIG. 17 is a screen used by the user to control the status of a video queue. 
FIG. 18 is a screen used by the user to control the operational settings of equipment 
associated with the user's computer. 

Detailed Description 

Introduction 

The features of a communication network may be described in terms of the following five 
parameters or qualities: 

1 . Immediacy 

2. Symmetry 

3. Network Structure 

4. Caching & Routing 

5. Extensibility 
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Immediacv 

A one-dimensional measure termed "immediacy" is an indication of the amount of time 
(possibly related to a level of effort) that elapses between a user sending a Video Message and a 
receiver receiving the Video Message. FIG. 1 depicts such a relationship between a method of 
5 sending messages called NetMeeting, that is known in the prior art, and another technology that 
is described in copending U.S. S.N. 09/497,587. On the left end of the diagram is "Real-time," 
meaning an absolute minimum lag time between video capture and viewing. On the right end of 
the diagram is "Deferred-time," meaning that the amount of lag time between video capture and 
viewing is unknown. The lag time can in principle be infinite (i.e., the video may never be 

10 viewed), for example because the intended recipient has no interest in viewing the video. 

NetMeeting is software that is available in version 3 as part of Windows 2000 from 
Microsoft Corporation. Information about Netmeeting can be found at Microsoft's website, 
www.microsoft.com. Netmeeting provides an immediate coupling between the sender of 
information and the receiver of the information. Once a messaging connection is made using the 

1 5 NetMeeting technology, the receiver renders the video sent by the sender. The receiver cannot 
turn off the video (or "opt-out" of viewing the video) except by completely terminating the 
NetMeeting session. The NetMeeting technology has the problematic features of being 
computationally intensive and requiring a large transmission bandwidth. The NetMeeting 
technology requires that the video must be processed in real time, because it is analogous to two- 

20 way television broadcasting over a network. The real time processing includes capturing the 
video, compressing the video, streaming the video, decompressing the video and rendering the 
video. Because all of these processes must be performed in real time, the NetMeeting real-time 
technology is beyond the capacity of the computers available to many non-professional or 
consumer users. 

25 In a "deferred-time" application such as the VideoShare Producer which is described in 

the copending U.S. S.N. 09/497,587, a receiver of the video message must formally acknowledge 
the receipt of the message and actively initiate the downloading and viewing process. For 
example, a potential viewer must activate his or her email browser, receive a new email message 
announcing the availability of the video, read the subject body, have interest in the email body 

30 content, activate the associated tag link (for example, by clicking on it), be connected to the 
World Wide Web ("Web"), and activate a "play" button in a viewer software application. 
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With some deferred-time applications, there are several stages at which the user can 
choose to ignore the message, or can avoid taking a necessary step to receive and view the video. 
The sequence of actions necessary to view a video may result in a receiver not viewing the video. 
Some commercial senders of videos, such as advertisers, may want to limit the possibility that a 
receiver might ignore or "opt out" of viewing a video. 
Symmetry 

Applications that involve the transmission of videos over a network, such as the Internet, 
generally use an asymmetric system design that divides the supporting technologies into two 
distinct groups: the video messaging "Sender" and the video messaging "Receiver." The 
"Sender" uses a sender computer provided with one set of software tools to capture, edit, and 
otherwise generate streaming video content. Prior art software products that perform the 
"sending" operations include the Microsoft Media Encoder and the Real G2 Producer. The 
VideoShare Producer of the copending U.S. S.N. 09/497,587 is another system and method for 
sending a streaming video from a sender to a server and then on to a viewer. The "Receiver" 
uses a receiver computer that is provided with a different set of software tools to receive, 
manipulate, and display the video. Prior art software products that implement the "receiver" 
functionality include the Windows Media Player and Real G2 Player. 

In asymmetric systems, the sending computer often cannot determine important features 
of the receiving computer. The receiving computer also often cannot determine important 
features of the sending computer. As an example, the sending computer generally does not have 
any information about the kind of software that the receiver computer has available. The sending 
computer generally cannot determine the capabilities of the receiver computer, such as CPU 
power or available network connection speeds. The sending computer generally knows nothing 
about the default control setting of the receiving computer as regards security protocols, preferred 
formats, and the like. In general, the application running on the sending computer and the 
application running on the receiving computer do not communicate directly. The "Sender" 
transmits a message for receipt by the "Receiver," without assurance that the "Receiver" is 
available to receive the message, or that the "Receiver" can decipher the message that it does 
receive. 

Furthermore, as a practical matter it is difficult, if not impossible, to extend the features 
of one of the "Sender" and the "Receiver" without a corresponding improvement or addition of 
capabilities by the other. Software for use in such applications is written and developed by 
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various organizations. In the absence of uniform standards, each developer can prepare software 
that has different features from the software of others. Some of the features can be proprietary. 
As one example, it is difficult to extend the features of both the video sending and receiving 
aspects at the same time. In such instances, a "Sender" and a "Receiver" may find that they 
5 cannot communicate. An example of such a failure to be able to communicate that is well 
known is the inability of some older Web browser software to correctly render Web pages 
prepared for use with different browser software because the embedded commands in the pages 
were not understood or were incorrectly interpreted by browsers other than the intended browser 
software. 

10 Network Structure 

The Video Share Producer system and method, which is described as one embodiment in 
copending U.S. S.N. 09/497,587, is a solution that depends on the operation of a server. FIG. 2 A 
depicts the structure of the VideoShare system and method as represented by the VideoShare 
Producer embodiment. A "sender A" using sender computer 302 generates or obtains a video 

15 that the "sender" wishes to share with or publish to a "receiver B." In one embodiment described 
in copending U.S. S.N. 09/497,587, sender computer 302 has a copy of the VideoShare Producer 
software that is used to obtain, prepare and transmit the video. In the VideoShare system and 
method, the sender computer 302 of "sender A" appears to have a direct connection to the 
receiver computer 304 of "receiver B." The dotted arrow from "A" to "B" denotes the apparent 

20 connection. In practice, the sender computer 302 of "sender A" transmits the video segment to a 
server 306 that is operated by a hosting service, such as the VideoShare service. This upload is 
denoted by the arrow extending from the sender computer 302 to the VideoShare server 306. 
The VideoShare server 306 in turn notifies the receiver computer 304 of "receiver B" that a 
video is available for viewing. Upon an acknowledgment from B that the video is desired by B, 

25 the VideoShare server 306 streams the video to the receiver computer 304 for viewing by B. The 
streaming of the video is denoted by the arrow extending from the VideoShare server 306 to the 
receiver computer 304. The receiver computer 304 uses software designed to display a streaming 
video, such as the Microsoft Media Player, for example, in order to display the video to "receiver 
B." 

30 In the VideoShare Producer system and method, when A, using sender computer 302, 

wants to send B, who uses receiver computer 304, a video message (for example, a Video 
Greeting Card), the apparent connection is between the computers 302, 304 of A and B directly. 
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However, the actual data connection is between A's sender computer 302 and the VideoShare 
server 306 and then between the server 306 and B's receiver computer 304. 

The steps in the process are outlined in overview in flowchart 201 in FIG. 2B 5 and as 
described in U.S.S.N. 09/497,587. At step 202, Sender A records a video message using sender 
5 computer 302 and associated hardware and software. At step 203, Sender A uploads the video 
message to VideoShare Server 306. In step 204, VideoShare Server 306 notifies Receiver B, 
who operates receiver computer 304, that a video message is available. In step 205, Receiver B 
acknowledges the notification (and requests delivery of the message). At step 206, the 
VideoShare Server 306 streams the video message to receiver computer 304, where Receiver B 

1 0 views the message. The apparent transaction, that of Sender A sending a video message to 
Receiver B, does not in fact occur directly, but only through the intermediation of the 
VideoShare Server 306, which is called on to both receive the video message and send on the 
video message. There is a computational load and transmission bandwidth requirement for each 
transmission of the video file. The operational costs, computational load, and bandwidth 

15 requirements to service a growing user base become ever larger as the number of users and the 
amount of traffic increases. 

In the model embodied in the VideoShare service, videos are uploaded to, stored on, and 
streamed from a server system, such as server computer 306, which can comprise one hosting 
facility at one location or multiple facilities at multiple locations. Each hosting facility can 

20 service a large number of concurrent users. The VideoShare Producer system and method can be 
considered topologically to be a "Star Network" where all clients machines that are operating in 
the system are in constant communication with a node, and all video traffic passes through the 
server. 

Such topologies have three main limitations. 

25 First, as the number of client computers concurrently using a server grows, the load on 

the single server node of the network increases rapidly. For example, in the system of FIG. 2, in 
which only A's sender computer 302 and B's receiver computer 304 are connected via server 
306, A can send a video only to B. However, if there are many additional users operating 
computers connected to the server 306, for example users C through Z, A can now send the same 

30 video to a plurality of recipients, namely B through Z. That is, the number of potential 

interactions (i.e., transmissions) increases at a rate far faster than the rate of increase of users. In 
the limit, the number of interactions is given by the theory of the number of combinations of N 
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things, where N is an integer greater than 1 , taken two at a time. This number increases at a rate 
of the order of the square of the number N of client computers and their users. As the load 
grows, the server's finite resources must be ever more thinly distributed to each client computer. 
In such a case, increased success in sharing video over a network requires more infrastructure to 
continue to perform the necessary tasks at an acceptable level of performance. 

Second, a failure on the single node of the network causes a large disruption of service to 
the connections of many client computers to the server. It is not uncommon that a single server 
problem can create communication problems for tens-of-thousands of client computers (and their 
users) at once. In addition, disturbances of a server that are less than complete failures can 
seriously degrade the service of many, for example by lengthening the time required to process 
requests that pass through the server. 

Third, in a network that has a server, the costs to build, to operate, and to maintain the 
server can be large. In particular, when the size of the typical message is large, such as is the 
case with video files, and the amount of traffic increases, the costs to maintain the ability of the 
server to perform at an acceptable level can escalate quickly. 
Caching & Routing 

A network that employs a server to handle data that flows between sender computers and 
receiver computers will in general require additional storage capacity in order to hold the 
messages that are in transmission to receiver computers that cannot immediately accept the 
messages. This situation involves having one or more caches or storage such as hard disks to 
hold the messages for later transmission from the server to the intended recipient computer. The 
server will then have to deal with the overhead of storing and retrieving messages in addition to 
the demands of receiving and sending the messages. If the server is also responsible for routing 
the messages sent between a sender computer and a receiver computer, the server will spend time 
communicating with other computers to determine whether sufficient bandwidth exists on a 
particular connection to carry the intended message, as well as time spent by the server in 
sending the message itself. 
Extensibility 

Extensibility refers to the ease or difficulty (and in the extreme, the very possibility) of 
expanding a system, for example by adding additional capacity and/or additional operational 
features. Normally, when a piece of software is released to the public, extensibility is 
accomplished through different versions that are sold separately as complete packages or through 
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upgrades. The user often is charged for upgrades to software packages. For example, version 1 .0 
of a word processor generally has fewer features than version 2.0 of the same software. The end 
user who wants to upgrade a software package must install the new version of software manually, 
typically either through an Internet download or through a purchase. 

With the advent of the Component Object Model (COM) in 1995, operating systems, 
such as the Windows operating systems, made available runtime binding of software objects. 
Runtime binding means that when the software application is run, a number of software objects, 
each comprising one core functionality, can become active and can create connections to one 
another. In the COM model, a particular object can be replaced with a different version of that 
object, with an associated change only in the function that the particular object provides. 
Therefore, in the Component Object Model, a software release is a collection of individual, 
replaceable, objects, rather than a single monolithic piece of software that must be reassembled 
in order to change some feature of the code. As long as software developers maintain a 
consistent Application Programming Interface (API), it is possible to exchange one version of a 
software object for another at runtime (i.e., perform an upgrade of a selected functionality). 
The Invention and the Attributes of Immediacy. Symmetry. 
Network Structure. Caching & Routing, and Extensibility 

The present invention, one embodiment of which is called the Instant Video Messenger 
(IVM), relates to methods and systems that provide advantages in Immediacy, Symmetry, 
Network Structure, Caching & Routing, and Extensibility. 
Immediacy 

FIG. 3 shows the relationship of an embodiment of the invention to the prior art 
NetMeeting technology and to the VideoShare Producer technology of the copending U.S.S.N. 
09/497,587. The embodiment is called the Instant Video Messenger (IVM). IVM falls into an 
intermediate position along the horizontal "immediacy" axis because the time for delivery from 
Sender A to Receiver B may range from nearly instantaneous to a lengthy interval, as will be 
described in more detail below. IVM is neither "Real-time" nor is it fully "Deferred-time". IVM 
allows content creators the ability (and the possibility) to send streaming video content with little 
delay to viewers with few chances to "opt-out." IVM can be used in conjunction with both free 
personal use or paid-for professional services. 
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Svmmetrv 

With the VideoShare Producer described in copending U.S. S.N. 09/497,587, functionality 
can be added to the Producer (e.g., video sending) and the web site (e.g., storage, notification, 
and the like). However, functionality generally cannot be added to the playback features that are 
available to a user who employs a third party playback application, such as the Windows Media 
Player, for example. 

Even if there is an agreement among software providers as to the features to be 
accommodated, there can still be "run-time" issues relating to compatibility. For example, a 
change of preferences or status by one of the sender computer and the receiver computer (e.g., 
changing connection speeds) needs to be communicated to the other, and there may be no 
mechanism to do so. For example, if a computer begins a new task, in particular in a 
multithreaded operating environment, the capacity available to those tasks that were running 
when the new task begins may have to be reduced. As an example, a communication link that 
exists when a new task begins may have to slow down to allow the computer's CPU to have an 
opportunity to handle the new task, and conversely, when a task ends, the CPU is freed of a 
requirement to service the terminating task, and can devote more effort to the communication, 
link which may then be capable of speeding up. 

According to one embodiment of the invention, the network configuration that supports 
the Instant Video Messenger is completely symmetric. The software running on the sender 
computer is exactly the same as the software running on the receiver computer, with the possible 
exception for differences due to version updates. The "Send/Receive" package can be prepared 
in a modular form comprising one or more Dynamic Link Library (.DLL) files, so that updating 
an installed copy of the software can be carried out by replacing an older version of a .DLL file 
with the most current version. The Instant Video Messenger is able to address the "knowledge" 
issues (i.e., the sender computer "knows" about the software operating on the receiver computer), 
because both computers operate the same software. The Instant Video Messenger is able to 
address the "feature extensibility" issues, (i.e., issues that relate to improvements that appear in 
both the sender software and the receiver software at the same time) because the software has a 
common origin, and is modular, thereby allowing updates of the corresponding transmit and 
receive modules at the same time. 
Network Structure 
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FIG. 4A shows a topology in accordance with one embodiment of the invention. 
According to the invention, the IVM client software that operates on a sender computer 302 has 
the capacity to transfer the contents of a video message directly to other IVM client software 
operating on a receiver computer 304. Each Sender/Receiver is capable of streaming video itself, 
without reliance on a network centralized server. In the IVM system servers have a more limited 
role than in the traditional client/server network model. In an IVM system, servers, such as the 
server 406, are responsible for database transactions, which require limited bandwidth and 
computational power in comparison to the higher bandwidth and computational requirements for 
streaming video from a server, in the traditional client/server model. In one embodiment, the 
server 406 has only limited database requirements. Such limited requirements for transmission 
bandwidth and for limited database manipulation simplify the replication of the server 406, 
because the server 406 need not be expensive or complicated. Ease of replication of the server 
406 is useful to provide redundancy in case of critical operational failure of the server 406, and to 
allow the addition of more such servers as the IVM system grows (i.e., more IVM client 
computers are added) and user demand for the IVM service/system increases. According to the 
present invention, the embodiment called the Instant Video Messenger does not rely on a video 
streaming center 306. 

FIG. 4B shows the steps in the process in overview in flowchart 401. At step 402, Sender 
A, using sender computer 302, asks the IVM Server 406 if Receiver B is online via receiver 
computer 304. IVM Server 406 responds that Receiver B is online. At step 403, Sender A 
records a video message using sender computer 302 and associated hardware and software. At 
step 404, Sender A, using sender computer 302, notifies Receiver B that a video message is ready 
for transmission to Receiver B via receiver computer 304. In step 405, Receiver B acknowledges 
the notification (and requests delivery of the message). At step 206, Sender A, using sender 
computer 302, streams the video message to receiver computer 304, where Receiver B views the 
message. The apparent transaction, that of Sender A sending a video message to Receiver B, 
occurs directly without the intermediation of the IVM Server 406, which not called on to either 
receive the video message or send on the video message. The exemplary process that occurs as 
described in FIG. 4A is distinct from the process as described in FIG. 2A, which requires the 
intermediation of the VideoShare Server 306 . 
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Caching & Routing 

The Instant Video Messenger dynamically routes video messages across its network. The 
IVM server 406 can determine whether any particular sender computer 302 and/or receiver 
computer 304 is/are on-line and available to send or receive video information (e.g., video files). 
The IVM server 406 also can re-route messages if network (e.g., Internet) traffic volume creates a 
problem with transmission between the sender computer 302 and the receiver computer 304. 

In addition, the IVM system can employ nodes that can store the messages that pass 
through them, enabling archiving applications, as described in more detail below. In one 
embodiment, a pool of video content can automatically build up by using such archiving 
applications. A node used to store such content can later be used to redistribute the material. 
Extensibility 

The Instant Video Messenger uses the COM technology to provide extensibility. When a 
new version of an object that uses the COM technology is developed, a user who has the old 
version is notified of the availability of the software object upgrade. The user can also be 
notified of the availability of new objects that provide additional functionality. For example, 
VideoShare has created a video effect COM API for use by software developers. This API 
provides an interface for the addition of new and improved video effects. Examples of video 
effects that are available in expensive, professional quality systems, and that can be provided in 
the IVM software, include cross-fading, titling, and blue-screening. As objects that incorporate 
or provide such functionality become available from the development community, they can 
immediately be available to the IVM end user, without the need for purchasing an upgrade or 
downloading the entire IVM software package. The IVM software thus provides the feature of 
extensibility. 

Since the Instant Video Messenger is symmetric, COM objects that provide new features 
can be prepared for both the sender computer (i.e., generation of material for transmission that 
enables a new feature) and for the receiver computer (i.e., the presentation of the material of the 
new feature to the recipient). To maintain complete symmetry, the upgraded COM objects for 
the sender and for the receiver are released together. Examples of new features include 
capabilities such as customized streams and meta-data that associate information with URL links 
and other ties to electronic commerce (e-commerce), as well as other hooks that enable a wider 
range of uses of video rather than just watching the images. A hook is a code segment that 
presently performs no useful function other than being a placeholder, but that can be substituted 
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with another COM object that performs a desired function. An example of a hook is a subroutine 
that includes only the command "return." These new and improved COM objects can be added 
to the IVM software at any time. 

The Instant Video Messenger permits, but does not limit, users to include additional 
information streams that are synchronous with the audio/video content. These peripheral 
synchronous information streams will be sent simultaneously with the primary audio and video 
streams. For example, the user can use these additional synchronous information streams to 
display business presentation slides. When the audio and video stream reaches a particular time 
location in the stream, for example when a CEO describes his or her business, a business 
presentation slide appears on the Instant Video Message receiver's computer. Another example 
of additional information streams associates "clickable-regions" (i.e., regions that are linked to 
an action that is activated by using a pointing device such as a mouse to highlight the region and 
then activating or clicking a mouse button) within a video frame, enabling the receiver of the 
Instant Video Message to bring up additional information (e.g., text descriptions such as a help 
screen, hyperlink URLs, and the like) by merely clicking on different areas of the video frame 
while watching the video. 
Instant Video Messenger Solution 

As shown in FIG. 4, the Instant Video Messenger uses a direct peer-to-peer connection 
between the computer 302 of A and the computer 304 of B. If A wants to send B a video 
message, the computer 302 of A sends the video content directly to the computer 304 of B. The 
video can be in either streaming or non-streaming format. This entire transaction is under the 
control of an Instant Video Messenger Server 406 (IVMS 406) that coordinates the run-time state 
of all members of the IVM Network (IVMN). The Instant Video Messenger Server 406 is 
notified when a given IVM user is on- or off-line, where the user is located (e.g., the user's IP 
address), what the user's Internet connections are (e.g. port address, firewall, proxy server 
configurations, and the like), and what the user's general activity has been (e.g., number of 
videos sent and/or received). The Instant Video Messenger Server 406 informs the sender 
computer 302 either that the receiver computer 304 is on-line (and a transmission is thus 
possible) or that the receiver computer 304 is not on-line. However, the Instant Video Messenger 
Server 406 does not have to deal with the details of, or the control of, a subsequent transmission. 

Furthermore, as depicted in FIG. 5, the IVM Server 406 is capable of sending messages, 
for example video messages, on its own account to the computers 302, 304 of both users A and 
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B. For example, after Sender A, using sender computer 302, streams a video message to 
Receiver B, using receiver computer 304, the IVM Server 406 can send its own video message 
(or messages) to either or both of Sender A and Receiver B via each user's computer 302, 304. 
Such a message can be, for example, video advertising, video promotion material, or other videos 
5 related to a particular theme as part of the services provided by the IVM Network. In one 

embodiment, the message sent by the IVM Server 406 appears in a window such as that depicted 
in FIG. 16, for example when either or both of the Sender A or the Receiver B is using Microsoft 
Outlook for e-mail service. 

As shown in FIG. 1 6, the Windows Media Player 910 is displayed with the message pre- 

1 0 loaded. The recipient of this embedded message only needs to activate the play button 920 on 

the Windows Media Player to see the video segment, rather than going to a URL hyper-link. The 
embodiment includes the conventional dialog boxes for entry of an email address for a recipient 
(box 902), a "carbon copy" ("cc") address (box 904), and a subject (box 906). In the 
embodiment shown, instructions are presented below the Windows Media Player 910 for the 

1 5 convenience of the recipient. This embodiment can also be used by a Receiver B to receive and 
view a message sent by Sender A. 

In one embodiment, depicted in FIG. 6, IVM users are grouped together by IVM 
Channels. IVM Channels can represent areas of commonality of interest, such as hobbies, 
business interests, sports interests, and the like. IVM Channels can be used to form a convenient 

20 set of well-organized users, for receipt of video messages such as advertisement, product/service 
offerings, and special offers (for example, special saver airline fares). In one embodiment, this 
feature of the IVM network allows the IVM Server 406 to deliver instant video messages to a 
well-organized base of users. The users who are members of a Channel may also use the 
Channel to contact each other. 

25 In accordance with the invention, a sender using sender computer 302 who desires to send 

a video message to a receiver using receiver computer 304 communicates with one or more IVM 
Channels IVC1 602, IVC2 604, IVC3 606, IVC4 608 in the set of IVM Channels 600 located on 
the network, such as the Internet, to find out if the receiver computer 304 is on-line. These 
communications are denoted by the bidirectional arrow that interconnects the sender A and 

30 receiver B with the set of IVM Channels 600. If the receiver computer 304 is on-line, the sender 
computer 302 can transmit the video message directly to the receiver computer 304, as denoted 
by the arrow from A to B. 
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The software, when operating on the receiver computer 304, performs a number of useful 
actions. The software can determine that the sender computer 302, or another computer, has sent 
a video message to it. In one embodiment, the software can receive and display the video 
message for the viewer of the receiver computer 304 without any activity or intervention by the 
5 viewer, and without the necessity of an action by a viewer of the receiver computer 304. The 
software can invoke other software modules to accomplish the display of the video message, 
again without any activity or intervention by the viewer, and without the necessity of an action by 
a viewer of the receiver computer 304. 

The software can issue a notification message to "Receiver B" (e.g., the operator or 

1 0 viewer of receiver computer 304) that a message is being sent to receiver computer 304. In one 
embodiment, the software can detect and display information about the sender and the message. 
The software can display for "Receiver B" any or all of an identifier of the sender of the video 
message, a subject of the video message, and a title of the video message. This information can 
be used by "Receiver B" to decide if, and when, he or she wants to view the video message. In 

1 5 this embodiment, the software can additionally present to a viewer of the receiver computer 304 
an opportunity to select one of accepting the streaming video message, and rejecting the 
streaming video message. When the viewer makes a choice, the software accepts the response of 
the viewer and processes the message in accordance with the response of the viewer. As an 
additional feature, the software can present to the viewer any combination of choices that 

20 includes rejecting the streaming video message and at least one of accepting the streaming video 
message for immediate display, and accepting the streaming video message for storage in a 
machine-readable memory, for display at a later time. In response to a selection of the choice of 
accepting the message for storage in a machine-readable memory, the software can, for example, 
direct the video to be saved on a machine-readable medium, such as a hard drive or any machine- 

25 readable medium that makes the video message available to the viewer at a later time. 

The machine-readable medium can be, for example, a computer floppy disk, a computer 
hard drive, a magnetic tape or the like, a CD-ROM, computer memory such as static or dynamic 
RAM, ROM, PROM, EPROM or the like, and/or any other mechanism or medium for storing 
machine-readable files, instructions, data or software. In a network, the machine-readable 

30 medium can be physically attached to a computer different from one on which the data may be 
used, or the software may operate. For example, in a network, an archival copy of software can 
reside on one computer and a copy can be copied to another computer, where the copy is 
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executed or otherwise used. If transfer time is not an issue, as when a viewer of a video puts off 
viewing to a later time, a file containing data or information (such as a video, a text file, a 
database file, a spreadsheet template or the like) may reside on the same computer as the one that 
received the file, or on a different computer that stores the file for the convenience of the viewer. 

Each IVM client computer maintains a personal IVM Contact List for the user of the 
computer, which is a list of other IVM users that are allowed to send messages to and to receive 
messages from the user. A user employs one of several notification methods such as email or 
Instant Messaging forums such as ICQ to invite another individual to communicate. 
Furthermore, the user can invite other individuals who have computers connected to the network, 
but who are unregistered with the IVM Network and whose computers do not have installed IVM 
software, to become users through an install executable that is sent in an email notification. The 
receiver of the email message can agree to accept the message by opening the email data 
attachment. This email data attachment contains the initial software codes needed to install the 
Instant Video Messenger software on the receiver's machine. His or her computer automatically 
executes the installation program, and the most recent version of the IVM software is installed 
and prepared for immediate use. 

The installed software enables the transmission and receipt of message components, 
which can include, for example, 

1) Video stream 

2) Audio stream 

3) Thumbnail image 

4) Subject text 

5) Message body text 

6) Document attachments (file transfers such as WinWord, and the like) 

7) Power-point slide presentations 

8) Meta-data 

A video message includes at least a video stream. 

When a user operating a sender computer 302 wishes to send a video message to a 
receiver who operates a receiver computer 304 that is not on-line at that time, the Instant Video 
Messenger Server 406 (or an IVM Channel 602, 604, 606, 608) allows the sender to transmit a 
message for later reception and viewing by the receiver. FIG. 7 depicts an embodiment of a 
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system and method that provides such functionality according to the invention. One embodiment 
involves a connection with the VideoShare technology described in copending U.S. S.N. 
09/497,587. The sender computer 302 has a video message that the operator of the computer 302 
wishes to transmit to the receiver computer 304. However, receiver computer 304 is offline. 
One of the IVM Channel Servers 602, 604, 606, 608 informs the sender computer 302 of this 
situation. The sender computer 302 is prompted by the IVM Channel Server 602 (for example) 
of the possibility of storing that video in the sender's VideoCenter supported by VideoShare 
server computer 710 for later delivery to the intended recipient. The sender computer 302 can 
send the video message for storage at server computer 710. Server computer 710 can transmit 
the video message, for example, via e-mail or via Web retrieval as described in copending 
U.S. S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again 
on-line. 

The Instant Video Messenger system includes a Instant Video Messenger Channel 
Provider Kit that enables third parties (e.g., affiliates, partners, co-developers and the like) to 
provide additional Instant Video Messenger Channels to the IVM community. The Instant Video 
Messenger Channel Provider Kit includes all of the components for hosting an IVM Channel on 
a server computer. The components include, but are not limited to, software, database 
connectivity, and communication protocols. The IVM Channel Provider dedicates a server 
computer to manage all of the communications between the server and the computers of the 
channel users (or channel members). The IVM Channel Provider installs database software such 
as Microsoft SQL 7 on the server to allow the server to handle a database that identifies the users 
and their computers. The IVM Channel Provider uses the database software to manage 
information to be communicated to the channel members, and configures the IVM Channel 
Provider software with an IVM Channel name and description. The IVM Channel Provider (or 
Host) can decide how it wishes to implement the service based on the expected demand for its 
IVM Channel services. 

These channels can be directed towards a particular customer segment, for example a 
segment based on thematic areas or common interests, such as for example, travel, auctions, or 
computers. As additional IVM Channels are developed by Instant Video Messenger Channel 
Providers, IVM users can be notified of their introduction. 



WO 01/10128 



PCT/US00/21214 



-23- 

Furthermore, the IVM Channel Provider Kit enables third party organizations to insert additional 
video streams before and/or after video messages that are sent from one user to another, 
including, but not limited to, advertisements in the form of video, still images, audio, and text. 

An alternative embodiment of the messaging systems and methods of the invention is 
depicted schematically in a flow diagram 800 shown in FIG. 8. The sender (i.e., Sender A) of the 
video message begins by launching the computer communication module 3000, known in this 
exemplary embodiment as the Videoshare 2Peer application software 3000, on his or her 
computer 302, as denoted at box 805. The sender uses the 2peer application software 3000, and 
so is also referred to as the "user" in FIG. 8. Using the sender computer 302, the user logs into a 
2peer channel, as indicated at box 810. A 2peer channel is an embodiment of an IVM channel, 
generally 600, of FIGs. 6 and 7. The user records a message comprising video and possible 
audio or other components, as indicated at box 815, which is reached from box 810 by travelling 
through Node A. In one or more optional steps, generally denoted by dotted box 820, the user 
can attach meta-data to the video message. At box 825, the user selects at least one recipient. 
The user can optionally select more than one recipient. For the purpose of the description, it is 
assumed that only one recipient is selected. However, if a plurality of recipients are selected, 
each recipient is handled individually, without regard for the situation of any other recipient. 

The user communicates, using the 2Peer application software 3000 and the sender 
computer 302, with the IVM server 406, and requests the online status of the intended recipient 
(i.e. Receiver B). IVM server 406 checks whether the receiver computer 304 of Receiver B is 
online via an IVM channel. The result of the inquiry is communicated to the sender. All of the 
request, the check, and the communication of results of the check is denoted by the box 830. 
Based on the online status of the receiver computer 304, either of two paths are selected. 

If the receiver computer 304 is online, corresponding to the arrow labeled "yes" exiting 
box 830, the Receiver B is notified, as indicated at box 835, using receiver computer 304 and a 
copy of the 2Peer application software 3000 thereon, that Sender A wishes to send a video 
message to Receiver B. At box 840, the Receiver B selects whether he or she wishes to accept 
the video message, decline the video message, or defer receipt of the message. If the receiver 
accepts the message, as denoted by box 845, the sender, using sender computer 302 and the 
2Peer application software 3000 thereon, steams the video to receiver computer 304, where the 
Receiver B can view the video message. The user is then carried in the system and method back 



WO 01/10128 



PCT/US00/21214 



-24- 

to Node A, where the process can be repeated, or the user can terminate the process as indicated 
by the box 850. 

If instead, the Recipient B declines the video message, the system and method operate by 

informing the user that the recipient has decline the message, as indicated at box 855. The user is 
5 then carried in the system and method back to Node A, where the process can be repeated, or the 

user can terminate the process as indicated by the box 850. 

In the third alternative, the Recipient B defers receipt of the message, as denoted by the 

arrow from box 840 to box 860. Box 860 indicates the action that the sender is notified that 

receipt of the message is being deferred by the recipient. 
10 If the recipient computer 304 is offline, the sender is informed that the receiver computer 

is not available to accept a video message, as indicated at box 865. 

In either the case of a recipient deferring acceptance of a video message, or the case of a 

receiver computer 304 that is not online, the systems and methods of the invention allow a 

substantially similar series of steps to take place. 
15 At box 870, the sender, using the sender computer 302 and the 2Peer application software 

3000, is asked by the IVM server 406 whether the sender wishes to store the message that cannot 

be delivered immediately. If the sender declines to store the message, the systems and methods 

of the invention carry the sender back to Node A, where the process can be repeated, or the user 

can terminate the process as indicated by the box 850. 
20 If the sender does wish to store the message, the sender's status as a registered member of 

the VideoShare service is checked, as indicated at box 875. If the sender is not a registered 

member of the VideoShare service, the sender is registered for the service, as indicated at box 

880. If the sender is a registered member, or has just registered, the sender logs on to the 

VideoShare service, as denoted at box 885. The sender then transfers the video message, using 
25 sender computer 302, to the VideoShare server 710, as indicated at box 890. The systems and 

methods of the invention carry the sender back to Node A, where the process can be repeated, or 

the user can terminate the process as indicated by the box 850. 

As indicated ay box 895, when the VideoShare server 710 determines that the receiver 

computer 304 of Recipient B is online, it streams the video message file to the receiver computer 
30 304 in a manner that is transparent to the sender, who may not be online when such streaming 

occurs. 
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Sample Applications 

The rich feature set and wide range of video and audio messaging is also suited to a 
corporate environment, particularly enabling remote communications such as intra-office video 
memos as well as messages sent between remote and home offices. The Instant Video 
5 Messenger can, for example, be used by co-workers to keep in touch with each other while 

working on a project, including such activities as sending text, audio, and video messages to one 
another while sharing collaborative documents. 
Implementation Details 

The entire Instant Video Messenger software system can be implemented in C++ within 
1 0 the Microsoft Foundation Class (MFC), ATL/COM, and ActiveX application frameworks. Each 
functional component (capture, compression, saving, contact list, etc.) can be a separate .DLL 
file. Such modularity allows easy upgrading, as users only have to download the components 
that have changed (generally files measured in the 10s of Kbytes). 

The Instant Video Messenger software is distributed and installed on a user's computer 
1 5 that has a connection, either temporary or permanent, to a network, such as, but not limited to, 
the Internet. The user may have a video capture device, such as a Web Camera or a video 
digitizing card, but it is not required for operation of the Instant Video Messenger. Users that do 
not have a video source can still elect to load in pre-existing media files and can send those 
media files as Instant Video Messages. 
20 Communications between each member of the communication session and the Instant 

Video Messenger Network can be performed via Transmission Control Protocol/Internet 
Protocol (TCP/IP) socket connections or User Datagram Protocol (UDP) connectionless 
datagrams. The user can select which TCP/IP or UDP network Port to use for communications 
as well as video streaming. 
25 In one embodiment according to the invention, the software structure is organized as a 

three-tier software system. The lowest level tier is a COM based set of functional components 
that implement the core functionality such as video capture, audio capture, network 
communication, and contact list management. The middle software tier exists as an interface 
layer which binds the lower level software tier to a graphical user interface such as windows, 
30 buttons, text, etc. This software layer is written as ActiveX components. The top software tier is 
written as an Microsoft Foundation Class (MFC) desktop application, grouping together the 
ActiveX components into a final product offering. This software installs itself (upon startup of 
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the computer) into the Windows Tool Bar as a small icon that can be opened and closed at the 
user's will. The software runs on the user's machine until it is shut down or manually 
terminated. 

Referring to FIG. 9, a user of the system, such as a private individual working from home, 
or a professional working from a business, employs a computer system 302. The user has an 
intention of sending a video message to a viewer of the message who is situated at another 
computer, that will be described in more detail below. The user of the computer system 302 may 
also be referred to as the "sender" or "Sender A", and computer system 302 may be referred to as 
the sender computer 302. The computer system 302 can include a computer which can be a 
personal computer of conventional type such as a desktop or laptop computer, a hand held device 
such as a PDA, or a more powerful computer such as a workstation, a server, a minicomputer, a 
mainframe, or the like. 

The computer system 302 can operate software including a web browser such as 
Microsoft Internet Explorer or Netscape Navigator or Communicator or the like, for 
communication over a network such as the Internet via the World Wide Web (hereinafter "the 
Web"), or to permit wireless communication. The computer system 302 can operate software 
that can manipulate video segment files. The computer system 302 can communicate with video 
sources, such a video cameras and video recording machines, if the user wishes to employ such 
sources. Conventional commercially available personal computers typically have sufficient 
capability to meet these requirements. The computer system 302 can also employ video 
segments generated digitally by the computer and appropriate software, or by another computer, 
if the user wishes to employ such techniques. In one embodiment, the computer system 302 
operates one or more modules that are part of a computer communication module 3000 called 
VideoShare 2Peer. 

The VideoShare 2Peer 3000 is a software application package that the user can download 
from the Web site www.2peer.com or that the user can obtain in other formats such as on a CD- 
ROM or bundled with other software or hardware. In the present invention, one or more 
modules that are components of the VideoShare 2Peer 3000 are present in software that the 
sender computer 302 controls. The one or more modules of the VideoShare 2Peer software can 
be operated by the user under his or her control on his or her computer, in the computer system 
302, in order to provide the capability of recording, converting, and optionally, compressing 
video segments, creating one or more identifiers for a video segment, and transmitting a video 
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segment with one or more of the identifiers to a receiver computer 304 when the receiver 
computer 304 is simultaneously online with the sender computer. If the receiver computer 3 04 is 
not online, the sender computer can optionally transmit the video segment to a host computer 60 
for storage at a location under the control of the host computer 60. The host computer 60 will be 
described further below. In addition, the software modules can provide the capability to make a 
request of a server computer 406 that server computer 406 determine the online status of a 
receiver computer 304 to which the sender would like to send a video message, and that the 
status be reported to the sender computer 302 so that the sender computer 302 can determine 
whether sending the video message directly to the receiver computer 304 independent of the 
server computer 406 is appropriate, or whether the video message should be sent to a host 
computer 60 for storage and for later transmission to the receiver computer 304. 

The computer in the computer system 302 of the user one can be connected to one or 
more kinds of equipment for generating video segments, such as a video camera such as a Web 
cam 12 or another type of video camera such as a professional quality video camera. The 
computer in the computer system 302 of the user can be connected to one or more kinds of 
equipment for providing prerecorded video segments, such as a video recorder 14, or another 
computer that can create digital video segments through the use of suitable software, such as for 
example digital video segments that have been created for various commercial films, or the like. 
Once the sender has obtained a video segment, and has manipulated it according to the 
procedures described below with regard to the operation of the software comprising the computer 
communication module, or its equivalent, the video segment with one or more identifiers is 
transmitted to the receiver computer 304 directly if the receiver computer 304 is online, or the 
video is sent the host computer 60 for storage and for retransmission to the receiver computer 
304 at a later time. 

The host computer 60 can include one or more server computers 62, 62' , 62" that 
communicate via a network such as the Web with other computers, such as the computer in the 
user's computer system 302. The one or more server computers 62, 62', 62" also communicate 
with a storage array 64, or optionally with a plurality of storage arrays substantially similar to 
storage array 64. The storage array 64 can be any convenient storage system, such as a redundant 
array of magnetic storage disks, one or more readable and writeable CD-ROMs, random access 
semiconductor memory, any combination of such storage devices, or the like. The host computer 
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60 can connect via the Web to one or more computers that comprise the Web, conceptually 
denoted by the box 70. 

A video segment that is stored and controlled by the host computer 60 may be delivered 
to and displayed for a viewer in a variety of formats, and through a variety of methods, as 
5 denoted generally by the box 80. In different embodiments, a video segment can be displayed as: 
a video greeting card 81, such as a person wishing another a happy birthday; as video email 82, 
as video that can be viewed on a remote website 83 (e.g., a video segment embedded into the 
remote website so that a viewer who visits the remote website sees the video segment as part of 
the page that is presented); as video commerce 84, for example a video that depicts a person 

1 0 describing his or her experience and training as part of a resume submitted on-line; or as a video 
advertisement 85, for example a video depicting the benefits or showing the use of a product. 
Many other like applications of the technology can be envisioned. The video segment can be 
made available to the viewer as a streaming video that is sent to the viewer. In some 
embodiments, the viewer can use a hand held device such as a PDA or a cellular telephone that 

1 5 can connect to a network such as the Internet to view the video segment. 

In FIG. 10, the computer 16 of the user's computer system 302 is shown. The box 18 is 
intended to schematically depict a user of a computer video input device, which device can be the 
computer 16 operating suitable software to generate digital video, or can be another such 
computer, or can be the web cam or video camera 12, or can be the video recording device 14, or 

20 the like. The user begins by producing and/or recording a video segment on the hard disk of the 
computer 16 or within the temporary memory of a handheld device. As a second step, the video 
segment of step 1 can optionally be compressed and /or can be changed as regards the computer 
file format in which it is recorded on the hard disk. As a third step, the sender uses the computer 
16 to request from the server computer 406 an indication of the online status of the receiver 

25 computer 304. As a fourth step, if the receiver computer 304 is determined to be online, the 
video segment recorded on the hard drive of the computer 1 6 is transmitted with one or more 
identifiers to the receiver computer 304. In different embodiments, the identifiers can include 
information comprising at least one of an identifier of the sender of the video message, a subject 
of the video message, and a title of the video message. The fourth step is schematically depicted 

30 by the arrow pointing generally from the computer 16 to the computer 90 of the viewer. The box 
92 is intended to schematically depict a user of a display device. In one embodiment, the display 
device can be the computer 90, or the display device can be a display device such as a Web TV, 
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or can be a video output device such as a television set with a suitable decoder, or the like. The 
display device can also be a wireless hand held device such as a PDA or a cellular telephone or 
the like. The streaming video segment can in one embodiment be delivered as part of a video 
greeting card 81. In an alternative embodiment, the video can be delivered as a streaming video 
5 directly to the viewer from the sender computer , without the viewer having to activate the 
receiver computer 304 . 

As shown in FIG. 1 1 , the user can obtain a copy of the VideoShare 2Peer software by 
downloading a copy of the software from the Website www.VideoShare.com 50, as indicated by 
the picture at numeral 1 . Alternatively, the user can obtain a copy of the VideoShare 2Peer 

10 software on machine readable media such as a CD-ROM or the like. The VideoShare 2Peer 
software can be bundled with one or more utility or application programs that are useful for a 
user to have, such as a "container" application so that the VideoShare 2Peer software can be 
operated on a desktop computer. The user can install the VideoShare 2Peer software on his or 
her computer 1 6and can register with the IVM service at no charge. In registering for the IVM 

1 5 service, the user obtains a username and a password that can be used to identify the user. The 
activity of installing the IVM software comprising the computer communication module on the 
user's personal computer or the like and registering with the IVM system is indicated by the 
picture at the numeral 2. 

In order to use the system, the user first obtains a video segment. The user can create the 

20 video segment, for example with a Web cam 12, or the user can use an existing video segment 
obtained from a video recorder 16, as indicated by the picture at the numeral 3. The VideoShare 
2Peer 3000 software includes a module that has direct capture capabilities that permit the user to 
create the video segment. 

The user can employ the VideoShare 2Peer 3000 software modules to optionally 

25 compress the video; to determine if a video segment is in a format that is compatible with 

streaming video; to convert the video to a file format that is compatible with streaming video if 
the video segment is not already in a file format that is compatible with streaming video; and to 
transmit the video segment together with one or more identifiers that represent selections that the 
user can make (for example, a still image selected from the series of images that comprise the 

30 video segment, and an identifier of the sender of the video segment (e.g., the user). The activities 
carried out in conjunction with the VideoShare 2Peer 3000 software modules are generally 
indicated by the graphic at numeral 4. 
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The video segment and the identifier(s) can be transmitted to the receiver computer 304 
directly if the receiver computer 304 is online, or if the receiver computer 304 is not online, can 
be transmitted to the host computer 60 for storage and for later distribution to the receiver 
computer 304. In one embodiment, the video message is transmitted in a streaming video file 
5 format. This transmission activity is denoted by the graphic at numeral 5. 

Upon a determination that the intended receiver computer 304 is online the host computer 
60 sends the video in streaming video format to the receiver computer 304, where a viewer can 
observe the video in real time. The activity of serving the video segment as a streaming video is 
denoted by the graphic at numeral 8. 

10 Alternatively, if the receiver computer 304 is not online, the sender computer 302 can 

send the video message to a host computer, as indicated at numeral 6, which host can then send 
the video on to the receiver computer when it is online, as shown at numeral 7. 

The majority of the VideoShare 2Peer software was developed as a Windows 95, 
Windows 98, and Windows 2000 ("Windows 9x/2000") compatible ActiveX control (e.g. an 

15 .OCX file), with additional components existing as active template library (ATL) component 
object model (COM) components that are instantiated during runtime. A "container 
application," named "2peer.exe," allows the VideoShare 2Peer ActiveX Control to be executed 
from the Windows 9x/2000 desktop. The VideoShare 2Peer Active X Control can also be 
embedded into a web page, as is done within the www.2peer.com web site. 

20 The custom written VideoShare 2Peer software includes the following binary/source code 

components: (1) VideoShare 2Peer ActiveX Control (2peer.ocx), (2) Streaming media rendering 
engine (vsrender.dll), (3) Streaming media network object (vsnetwork.dll), (4) Buddy-list 
Contact management (vscontact.dll), (5) 2Peer Channel management (vschannel.dll), (6) 
VideoCapture object (DirectShowRecord.dll and VFWrecord.dll), (7) VideoPlayback object 

25 (DSPlay.dll), and (8) Media management (vsmedia.dll) 

All components were entirely written by VideoShare Inc. The VideoShare 2Peer software 
is built upon the following third-party technologies that provide lower-level device support, 
document sharing, and file format conversion: (1) Microsoft's DirectShow; (2) Microsoft's 
Windows Media Technologies; (3) Microsoft's Video for Windows; (4) MAPI; AND (5) ICQ. 

30 When the user launches the VideoShare 2Peer software comprising the computer 

communication module, he or she will see the window depicted in FIG. 12 appear on his or her 
computer 1 6 operating the Win9x/2000 operating system. The login screen can be made optional 
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for repeat users by providing a unique identifier for the user, such as a password, or by installing 
on the user's computer or the like a record similar to the "cookies" used by some interactive 
computer systems operating on a network such as the Internet. 

When the user enters in his or her username in the box 410 labeled VideoShare Login 
Name and his or her password in the box 415 labeled VideoShare Password and activates the 
"Start VideoShare 2Peer" button 420, the VideoShare 2Peer 3000 software opens a TCP/IP 
socket connection to the VideoShare Upload/Database Server via port 80 in order to avoid 
typical Firewall and/or Proxy Server problems. If the box 430 labeled Remember password is 
checked, the VideoShare 2Peer 3000 software will remember the user's password, eliminating 
the necessity to type in that information each time the software is started. The VideoShare 
Upload/Database Server then verifies the validity of the usemame/password. Furthermore, the 
VideoShare 2Peer 3000 software will notify the user if there is a more recent version of the 
software available, giving him or her the opportunity to automatically download and install the 
new software. 

With this login dialog, the user can also receive help, by activating the "Help" button 450, 
taking the user to a web page on the VideoShare web site. The login dialog box can also be used 
to create a new VideoShare user account, by clicking the "Create Another Account" button 460. 

Once the login process has been completed, the VideoShare 2Peer 3000 software looks 
for available DirectShow audio and video capture devices. These available devices are 
enumerated and listed within the "Settings Tab" as described later. The VideoShare 2Peer 3000 
software initializes the audio and video capture device, by recalling as a default the device that 
was used most recently. 

VideoShare 2Peer Software Preview/Capture/Import Process 

After the capture device initialization, the VideoShare 2Peer software displays the 
window depicted in FIG. 13. 

The image 510 in the middle of the window is the video input stream from the initialized, 
default video capture source. The image in FIG. 13 is that of an employee of the assignee of the 
present invention, in the offices of the assignee. The VideoShare 2Peer software automatically 
builds a DirectShow "preview graph" where the video stream from the video device is displayed 
on the screen, but is not saved to disk. This gives the user the opportunity to adjust the camera, 
e.g. an opportunity to correct the camera position, the camera focus, the camera angle, the 
magnification of the image, and the like. 
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At the top of this window, the user is presented with five different "tabs", each presenting 
the user with different aspects of the VideoShare 2Peer software comprising the computer 
communication module. In FIG. 13, the tab labeled "Record/Playback" 520 is active, indicating 
that the VideoShare 2Peer software comprising the computer communication module is ready to 
5 acquire and/or display a video segment. 

At the bottom of the window, there is a status message 522 that displays the current 
operation of the VideoShare 2Peer software comprising the computer communication module. 
In FIG. 13, the status message 522 prompts the user to either activate the Record button 531 to 
create a new video segment, or to import an existing video segment by activating the Import 
10 Video button 535, both of which are described in more detail below. 

Directly below the video preview image 510 is a Capture/Playback Control Panel 530 that 
includes the following items: 

Record button 53 1 which begins a new audio/video capture; 
Stop button 532 which terminates an active audio/video capture operation; 
1 5 - Play button 533 which initiates the playing back of the last recorded or imported 

video; 

Delete button 534 which cancels the last record or import operation and begins a new 
video preview; 

Import Video button 535 which allows the user to select a pre-existing video file from 
20 his or her hard drive; 

Save and Share button 536, which in the present embodiment activates software 
modules that convert the current video file into a compressed streaming format, and 
that send the video; and 

Shuttle Bar 537 which is used to control the current position of the playback file 
25 together with forward button 537 and reverse button 538, allowing the user to rewind 

and fast forward through the current video. 
The software modules that operate upon the activation of Save and Share button 536 will be 
covered in a subsequent section in this document in detail. 

When the user begins to record a video, the VideoShare 2Peer software builds a new 
30 "Capture Graph" that renders the video stream to both the display window as well as to a 

temporary .AVI file on the user's hard drive. The audio/video capturing continues until the user 
activates the "Stop" button 532 at which point the VideoShare 2Peer software stops the "Capture 
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Graph", destroys the DirectShow filter, builds a Direct Show "Playback Graph", and displays the 
first frame of the captured video as video preview image 510. When the user activates the Play 
button 533 the DirectShow "Playback Graph" is put into running mode, playing back the entire 
recorded video from beginning to end. 

The user can also choose to import a pre-existing video, which in one embodiment can be 
a file format selected from the AVI, MPEG, or QuickTime file formats, by activating the Import 
Video button 535. The VideoShare 2Peer software automatically renders the correct 
DirectShow filter to display an imported video correctly. 
Save and Share Process 

Once a video segment has been recorded or imported into the user's computer 16 that is 
running the VideoShare 2Peer software, the user can choose to process the video segment with 
various optional alternatives by activating the Save and Share button 536. When the Save and 
Share button 536 is activated, the video segment is archived and distributed automatically. The 
VideoShare 2Peer software greatly simplifies the entire process by seamlessly automating the 
following steps that are depicted in FIG. 14 A: 

Video file format conversion, as required; 

Compression to a streaming multimedia format at a user-specified bitrate; 
Creating a "Thumbnail" JPEG snapshot of the video file, as an identifier that a user or 
a viewer can observe in order to assess the content of the video segment; 
if RC on line, sending video message to RC; 

if RC not on line, optionally sending the resultant video and thumbnail files to the 
VideoShare server computers 62, 62'; and 

Logging the transactions and managing the user's storage account, including causing 
the generation of an identification tag that the server computers 62, 62' can employ to 
retrieve the video segment for viewing. 
FIG. 14A shows a flow diagram 600 of an embodiment of the invention in which the 
VideoShare 2Peer software automates a number of steps in connection with uploading a video 
segment by activation of the Save and Share button 536 described in FIG. 13. As indicated at 
box 605, a user first obtains and selects a video segment for processing for distribution. The box 
605 schematically encapsulates all of the actions that a user takes as described in relation to 
FIGs. 4 and 5 above. When the user activates the Save and Share button 536 the actions 
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described below that are enclosed by the dotted line 607 are automatically carried out under the 
control of the VideoShare 2Peer software. 

The VideoShare 2Peer software subjects the selected video segment to analysis to 
determine whether the selected video segment is or is not in a file format that is compatible with 
a streaming video format, as indicated at diamond 610. Formats that are compatible with 
streaming media formats include formats such as MPEGs and QuickTime videos. If the selected 
video segment is not compatible with a streaming video format, it is converted to a compatible 
format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, 
"Convert to compatible file format." The conversion process performed by the VideoShare 
2Peer software creates a DirectShow filter graph that decompresses the video file into a 
temporary, uncompressed AVI file. 

The video segment file in a format that is compatible with streaming video is then 
temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 
16. This storing step is performed if the file was originally in a format compatible with 
streaming video by following the arrow marked "YES" that points from the diamond 610 to the 
box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was 
originally not in a format compatible with streaming video by following the arrow that points 
from the box 615 to the box 620. 

The stored temporary file representing the selected video is then analyzed by the 
VideoShare 2Peer software , as represented by diamond 625, "Should file be compressed?" to 
determine if the temporarily stored file should be compressed. If the software determines that the 
file should be compressed, as indicated by the arrow labeled "YES" that points from the diamond 
625 to the box 630, labeled "Compress file," the file is compressed. The compression involves 
compressing the video file to a user-specified bitrate, or the bandwidth that is required to view 
the video without disruption in the transmission. The user can select the desired bitrate via the 
"Settings Tab" that is described in more detail below. 

The file is then converted to a streaming multimedia format file as indicated by the box 
635, labeled "Convert file to streaming multimedia format ("SMF") file," as denoted by the 
arrow pointing from the box 630 to the box 635. If the file is not to be compressed, the flow 
follows the arrow labeled "NO" pointing from the diamond 625 to the box 635, and the file is 
then converted to a streaming multimedia format file as schematically represented by the box 
635. 
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The process that is performed by the VideoShare 2Peer software as denoted by the box 
635 involves reading in the video file, frame by frame, and converting the video into a streaming 
multimedia format. In one embodiment, the VideoShare 2Peer software uses the Windows 
Media Streaming Format, known as ASF or WMF, but it is not technologically restricted to this 
5 choice. The Windows Media Streaming Format comprises MPEG 4 v3 for the video stream and 
the Windows Media Audio format for the audio stream. The output of this file is stored as a 
temporary file on the user's hard drive, in one embodiment. 

The flow diagram indicates that the process makes a "thumbnail" of the video file, as 
represented schematically by the box 640, labeled "Create and temporarily store JPEG 

10 "thumbnail" identifier." The VideoShare 2Peer software produces a JPEG still image that is 
used as a reference image to the entire video file. It is an identifier of the subject matter or 
content of the video that a user or a viewer can readily recognize, as compared to an 
alphanumeric string such as a typical string used to identify a file by its drive, directory (and one 
or more subdirectories) and filename. Such alphanumeric identifiers are useful, but may be 

15 totally uninformative as to the content or subject matter contained in the identified file or video 
segment. In one embodiment, the VideoShare 2Peer software creates the "thumbnail" by taking 
the "middle" image of the entire video file, as measured by the temporal duration of the file. In 
another embodiment, the selection of an image from which to make the "thumbnail" can be left 
to the discretion of the user. This JPEG file is also stored as a temporary file on the user's hard 

20 drive, in one embodiment. 

The next part of the process is either sending the video message to the receiver computer 
304, if the receiver computer 304 is online, or optionally sending the video message to the host 
computer 60 if the receiver computer 304 is not on line. 

Using the symmetric software present on both the sender computer 302 and the receiver 

25 computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver 
computer 304 if the receiver computer 304 is online, as indicated at box 641 . As shown in Fig. 
4B, Sender A, using sender computer 302, asks IVM Server 406 if Receiver B is online via 
receiver computer 304. For explanatory purposes, we will assume that IVM Server 406 responds 
that Receiver B is online. Sender A records, or already has recorded, a video message using 

30 sender computer 302 and the associated hardware and software. Sender A, using sender 

computer 302, notifies Receiver B via receiver computer 304 that a video message is ready for 
transmission. Assuming that Receiver B wishes to accept the video message, Receiver B 
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acknowledges notification using receiver computer 304 and requests delivery of the message. 
Sender A, using sender computer 302, streams the video message to Receiver B via receiver 
computer 304. Using the symmetric software present on both the sender computer 302 and the 
receiver computer 304, the sender 40 (or Sender A) can send the video message directly to the 
receiver computer 304 if the receiver computer 304 is online, as indicated at box 641. 

In the event that the receiver computer 304 is not online, the IVM Server 406 informs 
sender computer 302 that the receiver computer 304 is not online. Rather than sending a 
message to the receiver computer 304, the sender computer 320 sends the video message to a 
VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and 
for later delivery to the intended recipient. Server computer 710 can transmit the video message, 
for example, via e-mail or via Web retrieval as described in copending U.S. S.N. 09/497,587, to 
the intended recipient computer 304 when that computer 304 is again on-line. These steps are 
shown in boxes 645, 650, 655, 660 and 670. 

The optional part of the process is the upload operation, in which the VideoShare 2Peer 
software contacts the host computer 60, which in one embodiment is the VideoShare 
Upload/Database Server at the VideoShare hosting facility. This portion of the automated 
process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and 
JPEG thumbnail identifier to host computer 60." The VideoShare 2Peer software notifies the 
host computer 60 that the user wishes to place his or her video into a repository maintained by 
the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a 
repository of all recorded and uploaded videos to date. This upload is performed automatically 
via a direct TCP/IP socket connection over a specific connection port of the user's computer 
known as port 80. The VideoShare 2Peer software uses a standard communications protocol to 
perform this transfer to the host computer 60. In another embodiment, a proprietary protocol can 
be used, for example if one wants to maintain the security of information contained in the video 
segment. In another embodiment, the video segment can be encrypted in order to provide 
enhanced security. Both the compressed video streaming multimedia file and the thumbnail 
image are uploaded at substantially the same time. 

As schematically depicted by box 650, labeled "Delete temporary file to conserve storage 
space on user's computer," the VideoShare 2Peer software removes all of the temporary files that 
were created in the course of the automated processing described above. This feature provides 
for the user a convenient, secure, and transparent process, with the benefit that the user's 
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computer storage device(s), for example one or more hard drives, do not become cluttered with 
unnecessary and obsolete files. 

In the event that the video message has been sent to the host, the VideoShare 2Peer 
software and the host computer 60 (for example, the VideoShare Upload/Database Server) will 
5 update the user's account to account for the required storage space that the video requires. The 
necessary logging, creation of an identification tag, and storing of the video and the associated 
identifier or identifiers is also performed automatically, as schematically depicted by box 655. 

The user can optionally add additional identification and control information about the 
user, and about how and under what conditions the video is to be made available for distribution, 

1 0 as schematically indicated by box 660. The user is automatically prompted to provide this 

information, but has the option to forego making a decision immediately. The transmission of 
video segment files to viewers is discussed in more detail below, and is represented in FIG. 14A 
by the box 670 labeled "Transmit file to viewer" which is outside the region 607 as an indication 
that the transmission of files to viewers is an action beyond the material discussed above in 

15 conjunction with the Save and Share button 536 of FIG. 13. 

FIG. 14B shows a flow diagram 601 of another embodiment of the invention in which 
software automates a number of steps in connection with uploading a video segment. Many of 
the steps already described in connection with FIG. 14A also occur in the embodiment depicted 
in FIG. 14B, and are numbered in the same manner as in FIG. 14 A. In FIG. 14B, there is first an 

20 optional step indicated by the box 604 labeled "Optional: User authentication with server" in 
which the User is optionally required to provide identification, such as a user name and 
password, that authenticates the identity of the user to the server or host computer 60. The user 
then obtains and selects a video segment for processing for distribution, as indicated at box 605 
that schematically encapsulates all of the actions that a user takes as described in relation to 

25 FIGs. 4 and 5 above. When the user activates the Save and Share button 536 the actions 

described below that are enclosed by the dotted line 608 are automatically carried out under the 
control of the VideoShare 2PeerSoftware comprising the computer communication module. 

As discussed in relation to FIG. 14A, the VideoShare 2Peer software subjects the selected 
video segment to analysis to determine whether the selected video segment is or is not in a file 

30 format that is compatible with a streaming video format, as indicated at diamond 610. If the 
selected video segment is not compatible with a streaming video format, it is converted to a 
compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to 
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the box 615, "Convert to compatible file format." The conversion process performed by the 
Video Share 2Peer software creates a DirectShow filter graph that decompresses the video file 
into a temporary, uncompressed AVI file. 

The video segment file in a format that is compatible with streaming video is then 
5 temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 
1 6. This storing step is performed if the file was originally in a format compatible with 
streaming video by following the arrow marked "YES" that points from the diamond 610 to the 
box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was 
originally not in a format compatible with streaming video by following the arrow that points 

1 0 from the box 6 1 5 to the box 620. 

The stored temporary file representing the selected video is then analyzed by the 
VideoShare 2Peer software, and optionally compressed as represented by the box 623 labeled 
"Optional compression of file." The file is then converted to a streaming multimedia format file 
as indicated by the box 635, labeled "Convert file to streaming multimedia format ("SMF") file." 

1 5 Alternatively, a file from the box 620 can be uploaded to the host computer 60 without being 
converted to a streaming format, and the conversion to a streaming video format can be 
accomplished at the host computer 60. The process that is performed by the VideoShare 2Peer 
software as denoted by the box 635 involves reading in the video file, frame by frame, and 
converting the video into a streaming multimedia format. 

20 The flow diagram indicates that the process makes a "thumbnail" of the video file, as 

represented schematically by the box 640, labeled "Create and temporarily store JPEG 
"thumbnail" identifier." 

Using the symmetric software present on both the sender computer 302 and the receiver 
computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver 

25 computer 304 if the receiver computer 304 is online, as indicated at box 641 . 

In the event that the receiver computer 304 is not online, the IVM Server 406 informs 
sender computer 302 that the receiver computer 304 is not online. Rather than sending a 
message to the receiver computer 304, the sender computer 320 sends the video message to a 
VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and 

30 for later delivery to the intended recipient. Server computer 710 can transmit the video message, 
for example, via e-mail or via Web retrieval as described in copending U.S. S.N. 09/497,587, to 
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the intended recipient computer 304 when that computer 304 is again on-line. These steps are 
shown in boxes 645, 650, 655, 660 and 670. 

The optional part of the process is the upload operation, in which the VideoShare 2Peer 
software contacts the host computer 60, which in one embodiment is the VideoShare 
Upload/Database Server at the VideoShare hosting facility. This portion of the automated 
process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and 
JPEG thumbnail identifier to host computer 60." The VideoShare 2Peer software notifies the 
host computer 60 that the user wishes to place his or her video into a repository maintained by 
the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a 
repository of all recorded and uploaded videos to date. This upload is performed automatically 
via a direct TCP/IP socket connection over a specific connection port of the user's computer 
known as port 80. The VideoShare 2Peer software uses a standard communications protocol to 
perform this transfer to the host computer 60. In another embodiment, a proprietary protocol can 
be used, for example if one wants to maintain the security of information contained in the video 
segment. In another embodiment, the video segment can be encrypted in order to provide 
enhanced security. Both the compressed video streaming multimedia file and the thumbnail 
image are uploaded at substantially the same time. 

The next part of the process is the upload operation, in which the VideoShare 2Peer 
software contacts the host computer 60, which in one embodiment is the VideoShare 
Upload/Database Server at the VideoShare hosting facility. This portion of the automated 
process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and 
JPEG thumbnail identifier to host computer 60." Both the compressed video streaming 
multimedia file and the thumbnail image are uploaded at substantially the same time. 

As schematically depicted by box 650, labeled "Delete temporary file to conserve storage 
space on user's computer," the VideoShare 2Peer software removes all of the temporary files that 
were created in the course of the automated processing described above. This feature provides 
for the user a convenient, secure, and transparent process, with the benefit that the user's 
computer storage device(s), for example one or more hard drives, do not become cluttered with 
unnecessary and obsolete files. 

In the event that the video message has been sent to the host, the VideoShare 2Peer 
software and the host computer 60 (for example, the VideoShare Upload/Database Server) will 
update the user's account to account for the required storage space that the video requires. The 
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necessary logging, creation of an identification tag, and storing of the video and the associated 
identifier or identifiers is also performed automatically, as schematically depicted by box 655. 

The user can optionally add additional identification and control information about the 
user, and about how and under what conditions the video is to be made available for distribution, 
5 as schematically indicated by box 660. The user is automatically prompted to provide this 

information, but has the option to forego making a decision immediately. The transmission of 
video segment files to viewers is discussed in more detail below, and is represented in FIG. 14B 
by the box 670 labeled "Transmit file to viewer" which is outside the region 608 as an indication 
that the transmission of files to viewers is an action beyond the material discussed above in 

10 conjunction with the Save and Share button 536 of FIG. 13. 

FIG. 14C shows a flow diagram 602 of an embodiment of the invention in which 
software automates a number of steps in the formatting of a video segment. In particular, in this 
embodiment, the video segment that the user wishes to provide in streaming video format is 
compressed into a plurality of formats, each of which is encoded for optimal display at a different 

15 transmission bitrate. There can be a benefit to recording the same video segment in multiple 
formats. For example, a casual viewer may have only a slow speed modem, such as a 28.8 
kilobaud (kB) modem. For such a viewer, the slow transmission speed can make the size of a 
file a critical feature. Such a user can view a video in real time if it is formatted for a 28.8 kB 
modem, but not if it is formatted for appreciably higher transmission speeds. Another user, for 

20 example, one who has a Tl connection that can handle transmission speeds up to approximately 
1.5 megabaud, could successfully receive a version of the same video segment that is formatted 
for higher transmission speeds, with the possibility of having a better quality image and higher 
resolution, perhaps with better audio as well. The Tl user could see the version of the video 
segment intended for 28.8 kB transmission if he or she wanted to, but might prefer to see a video 

25 segment that appeared to be more professional in quality. By using a system that can 

automatically discriminate the transmission speed capabilities of the hardware that the user 
employs, the embodiment allows each user to view a version of the video segment that is 
optimally configured for the user's hardware. 

In particular, the steps of the method enclosed within the dotted rectangle 609 are 

30 automated by software that embodies the present invention. As described above, the user obtains 
and selects a video segment for processing for distribution, as indicated at box 605 that 
schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 
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and 5 above. When the user activates the Save and Share button 536 the actions described below 
that are enclosed by the dotted line 609 are automatically carried out under the control of the 
VideoShare 2PeerSoftware comprising the computer communication module. 

As discussed in relation to FIG. 14A, the VideoShare 2Peer software subjects the selected 
video segment to analysis to determine whether the selected video segment is or is not in a file 
format that is compatible with a streaming video format, as indicated at diamond 610. If the 
selected video segment is not compatible with a streaming video format, it is converted to a 
compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to 
the box 615, "Convert to compatible file format." The conversion process performed by the 
VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file 
into a temporary, uncompressed AVI file. 

The video segment file in a format that is compatible with streaming video is then 
temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 
1 6. This storing step is performed if the file was originally in a format compatible with 
streaming video by following the arrow marked "YES" that points from the diamond 610 to the 
box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was 
originally not in a format compatible with streaming video by following the arrow that points 
from the box 615 to the box 620. 

The temporarily stored file is then compressed in multiple streaming multimedia formats, 
as denoted by the box 633. In the present example, three files will be used to describe the 
process, but it should be understood that more or fewer than three formats may be created at 
substantially the same time. The resulting multiple files are denoted by the three boxes 634, 636 
and 638 labeled "Bandwidth Target A," "Bandwidth Target B," and "Bandwidth Target C," 
respectively. Each file is optimally encoded for play as a streaming video segment at a particular 
transmission rate and bandwidth, such as 28.8 kB, 56 kB, lOOkB, 300kB, or other transmission 
rates. 

As described above, the method includes a step of creating and temporarily storing a 
"thumbnail" identifier, as denoted by the box 640. Rather than transmitting one video segment 
in one SMF with one thumbnail, the embodiment of FIG. 14C transmits all the files 634, 636 and 
638 in association with the single thumbnail and any other identifiers that are selected as 
appropriate. For example, each SMF file can be identified as to its bandwidth. In an alternative 
embodiment, the system transmits only a single SMF file with its associated identifiers, including 
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the JPEG "thumbnail," and the multiple bandwidth variants of the SMF file are generated at the 
host computer 60. This embodiment may be advantageous when the user has only a slow speed 
modem, and would be severely time constrained by having to upload multiple files. 

Using the symmetric software present on both the sender computer 302 and the receiver 
computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver 
computer 304 if the receiver computer 304 is online, as indicated at box 641. 

In the event that the receiver computer 304 is not online, the IVM Server 406 informs 
sender computer 302 that the receiver computer 304 is not online. Rather than sending a 
message to the receiver computer 304, the sender computer 320 sends the video message to a 
VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and 
for later delivery to the intended recipient. Server computer 710 can transmit the video message, 
for example, via e-mail or via Web retrieval as described in copending U.S. S.N. 09/497,587, to 
the intended recipient computer 304 when that computer 304 is again on-line. These steps are 
shown in boxes 645, 650, 655, 660 and 670. 

The remaining steps of this embodiment, as denoted by the boxes 650, 655, 660 and 670, 
correspond substantially to the steps in FIG. 14A represented by the boxes identified with the 
corresponding numerals. It should be noted that the precise order of some of the steps, for 
example, the step denoted by the box 655 and the step denoted by the box 650, can be 
interchanged without a different outcome of the overall process. Other such interchanges in 
sequence are possible as well, again without a different outcome of the overall process. 

FIG. 14D depicts an embodiment of the database 64 of the host computer 60 on which are 
recorded the three exemplary bandwidth target files 634, 636 and 638 for FIG. 14C. These files 
are available for delivery over a computer network to a viewer. The files 634, 636 and 638 
represent three versions of the same video segment in streaming multimedia format, each suitable 
for optimal viewing by a user having hardware operating at the transmission rate corresponding 
to the format of one of the files. 

As shown in FIG. 14E, the user (or the viewer) transmits to the host computer 60 a 
request for a particular video segment, denoted by the arrow from the box labeled "USER" to the 
box 960 labeled "Connection Speed Detector." Host computer 60 can include hardware that can 
sense the transmission speed of a user computer 16, or of a computer used by a person desiring to 
view a video segment. Alternatively, the host computer 60 can inquire of the computer on the 
network that is connected to the user computer 1 6 or the computer of a viewer about the speed of 
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connection that is being maintained. When the information is available to the host computer 60, 
the host computer 60 can determine which file of the files exemplified by 634, 636 and 638 is 
most appropriate to serve to the user or viewer, as denoted by the box 692 labeled "Logic to 
select and serve SMF file to User." The host computer 60 then transmits the appropriate file to 
5 the user, as denoted by the arrow from the box 692 to the box 694 labeled "User receives and 
views SMF file." Alternatively, the viewer can request the transmission of a file encoded at a 
specific bitrate. 

When the user begins the process described in relation to FIG. 14A, in one embodiment, 
the "Progress Dialog" screen 700 depicted in FIG. 15 is presented, reflecting the status of the 
10 process in real time. The "Progress Dialog" screen 700 notifies the user about the total number 
of bytes that have to be uploaded to perform the transfer and it also informs the user of the 
number of bytes and the percentage of the file that have been uploaded in real time. 

FIG. 16 shows one embodiment of a screen, discussed above, that displays a video 
message. 

15 FIG. 17 shows a screen 1000 used to control the status of a video queue. When the user, 

after recording or importing a video, clicks the "Save and Share" button 536 of FIG. 13 while in 
"offline mode", the VideoShare 2Peer software performs the first three steps of the "Save and 
Share Process," namely, the video file format conversion represented by box 615 of FIG. 14A, 
the compression of the video segment to a streaming multimedia format at a user-specified 

20 bitrate represented by the box 635 of FIG. 14 A, and the creation of a "Thumbnail" JPEG 

snapshot of the video file represented by the box 640 of FIG. 14A. The resulting output files are 
stored in a local database for later use in the "Sharing Queue," which is an operation similar to 
the temporary storage of files depicted in FIG. 14A. In the middle of FIG. 17 is a dialog box 
1010 that displays a list of video segments that are ready to be uploaded to the VideoShare Web 

25 site. The small "Preview" window 1020 in the upper left corner of FIG. 17 is a DirectShow 

playback graph that allows the user to review the stored video segment that is highlighted in the 
dialog box 1010. The user can use this window to preview the video segment file by activating 
the "Preview" button 1030, to delete the video segment file by activating the "Delete" button 
1040, and to upload and publish the video by activating the "Save and Share Now" button 1050. 

30 The "Save and Share Now" button 1 050 performs the uploading process on each of the 

queued videos, creating a TCP/IP connection to the VideoShare Upload/Database Server, 
transferring the file to the VideoShare web site, and updating the user's VideoShare account, in a 
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manner substantially similar to the method employed by the Save and Share button 536 of FIG. 
13 to accomplish the same activities. 

Audio/Video Setting Process 

5 FIG. 1 8 shows a screen 1 100 used to control the operational settings of equipment 

connected to the user's computer. Another feature of the VideoShare 2Peer software the ability 
of the user to change the configuration of the audio, video, and compression devices through the 
use of the "Settings" tab 1110. Upon activation of the Settings tab 1110, the screen 1 100 is 
active. 

10 The user can select the "bitrate" at which the streaming multimedia files will be 

compressed by using the set of radio buttons 1 120 at the upper left corner of the screen 1 1 00. 
The default setting is "56k Modem" which corresponds to a user using a 56k modem. This 
default setting is denoted by the 56k Modem radio button 1 120 appearing with a dot, while the 
remaining radio buttons for bitrate 1 120 are blank. In one embodiment, the pie graph 1130 that 

1 5 appears at the upper right corner of screen 1 1 00 indicates the percentage of the user's 
VideoShare storage space that is full. In the embodiment shown, the user has filled 
approximately 3.13% of the available storage capacity available for storing files. Two pull-down 
menus, "Camera source device" box 1 140 and "Audio source device" box 1 150, list all of the 
available video and audio capture sources that the user has available on his or her Win9x/2000 

20 machine. The user can select a source of audio or video by activating the appropriate pull-down 
menu box and locating a device of his or her choosing. To the right of these pull-down menus, 
there are two buttons, "Video Settings. . ." 1 1 60 and "Audio Settings. . ." 1 170 that allow the user 
to change the properties of the currently selected audio and video device. Such properties 
include image size, capture compression, lighting conditions, and the like. The screen 1 100 also 

25 provides to the user the current working directory information in a the box 1 180 and the current 
queue directory information in the box 1 190, which the user can optionally change by entering 
new values in either or both boxes 1 180 and 1 190. 

Equivalents 

30 While the invention has been particularly shown and described with reference to specific 

preferred embodiments, it should be understood by those skilled in the art that various changes in 
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form and detail may be made therein without departing from the spirit and scope of the invention 
as defined by the appended claims. 
What is claimed is: 
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Claims 

1 1 . A method of sending a streaming video message over a network, comprising the steps of: 

2 receiving at a server computer an indication that the operator of a sender computer 

3 intends to send a streaming video message to a receiver computer over a network; 

4 determining whether said receiver computer is online; and 

5 communicating from said server computer to said sender computer whether said receiver 

6 computer is online, wherein an indication that said receiver computer is online signals to said 

7 sender computer that streaming said video message directly from said sender computer to said 

8 receiver computer over the network is appropriate. 

1 2. The method of claim 1, wherein the step of receiving at a server computer an indication 

2 that the operator of a sender computer intends to send a streaming video message to a receiver 

3 computer is performed by a supervisory computer communication module operable on said 

4 server computer. 

1 3. The method of claim 1, wherein the step of determining whether said receiver computer 

2 is online is performed by a supervisory computer communication module operable on said server 

3 computer. 

1 4. The method of claim 1, wherein the step of communicating from said server computer to 

2 said sender computer whether said receiver computer is online is performed by a supervisory 

3 computer communication module operable on said server computer. 

1 5. A method of sending a streaming video message over a network, comprising the steps of: 

2 sending to a server computer an indication that the operator of a sender computer intends 

3 to send a streaming video message to a receiver computer over a network; 

4 receiving at said sender computer an indication of whether said receiver computer is 

5 online; and 

6 in response to an indication that said receiver computer is online, streaming said video 

7 message from said sender computer to said receiver computer over the network independent of 

8 the operation of said server computer. 

1 6. The method of claim 5, wherein the step of streaming said video message from said 

2 sender computer to said receiver computer comprises: 

3 providing on said sender computer a computer communication module operable on each 

4 of said sender computer and said receiver computer; and 
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5 streaming said video message from said communication module of said sender computer 

6 to said receiver computer over the network independent of the operation of said server computer. 

1 7. The method of claim 6, wherein providing a computer communication module operable 

2 on each of said sender computer and said receiver computer comprises providing individual 

3 copies of the same communication module to at least one of the sender computer and the receiver 

4 computer, each copy operable on each of said sender computer and said receiver computer. 

1 8. The method of claim 7, wherein the step of providing individual copies of the same 

2 communication module comprises providing individual copies of the same communication 

3 module, said communication module being symmetric in operation. 

1 9. The method of claim 8, wherein the step of providing individual copies of the same 

2 communication module, said communication module being symmetric in operation comprises 

3 providing communication modules that are modular. 

1 10. The method of claim 8, wherein the step of providing individual copies of the same 

2 communication module, said communication module being symmetric in operation comprises 

3 providing communication modules that can determine the features present in another 

4 communication module operating on another computer. 

1 11. The method of claim 5, wherein the step of streaming said video message from said 

2 sender computer to said receiver computer over the network independent of the operation of said 

3 server computer comprises performing said streaming operation using only communication 

4 modules present on said sender computer and said receiver computer. 

1 12. The method of claim 5, further comprising the step: 

2 in response to an indication that said receiver computer is not online, optionally sending 

3 said video message from said sender computer to a host computer, for storage and later 

4 transmission to said receiver computer, independent of the operation of said server computer. 
1 13. The method of claim 5, wherein said video message comprises video information. 

1 14. The method of claim 13, wherein said video message further comprises information 

2 selected from audio information, a thumbnail image, a text identifying a subject, a message body 

3 text, a document attachment, a slide and meta-data. 

1 15. A computer program recorded on a machine-readable medium, the computer program 

2 adapted to transmit and receive streaming video messages over a network independent of a server 

3 computer, the computer program when operating on a sender computer performing the steps of: 
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4 



5 



requesting from a server computer the status of a receiver computer, said receiver 
computer being the intended recipient of a video message; 



6 



receiving an indication of the online status of said receiver computer; and 

in response to an indication that said receiver computer is online, streaming said video 



7 



8 message from said computer program of said sender computer to said computer program of said 

9 receiver computer independent of the operation of said server computer. 

1 16. The computer program recorded on a machine-readable medium of claim 15, wherein 

2 said computer program is further capable, when operating on a receiver computer, of performing 

3 the step of: 

4 determining that a video message is being sent by a sender computer. 

1 17. The computer program recorded on a machine-readable medium of claim 1 6, wherein 

2 said computer program is further capable, when operating on a receiver computer, of performing 

3 the step of: 

4 displaying to a viewer of said receiver computer information about said video message, 

5 said information comprising at least one of an identifier of the sender of said message, a 

6 subject of said message, and a title of said message. 

1 18. The computer program recorded on a machine-readable medium of claim 15, wherein 

2 said computer program is further capable, when operating on a receiver computer, of performing 

3 the steps of: 

4 receiving said streaming video message; and 

5 displaying said streaming video message to a viewer of said receiver computer. 

1 19. The computer program recorded on a machine-readable medium of claim 1 8, wherein the 

2 step of receiving said streaming video message comprises; 

3 receiving said streaming video message; 

4 presenting to a viewer of said receiver computer an opportunity to select one of: 

5 (i) accepting the streaming video message; and 

6 (ii) rejecting the streaming video message; 

7 accepting a response of said viewer; and 

8 processing said message in accordance with the response of said viewer. 

1 20. The computer program recorded on a machine-readable medium of claim 1 9, wherein the 

2 step of accepting the streaming video message comprises at least one of; 

3 (i) accepting the streaming video message for immediate display; and 
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4 (ii) accepting the streaming video message for storage in a machine-readable memory 

5 for display at a later time. 

1 21 . The computer program recorded on a machine-readable medium of claim 15, wherein 

2 said computer program is further capable, when operating on a receiver computer, of performing 

3 the step of: 

4 receiving said streaming video message is accomplished without the necessity of an 

5 action by a viewer of said computer. 

1 22. The computer program recorded on a machine-readable medium of claim 2 1 , wherein 

2 displaying said streaming video message is accomplished without the necessity of an action by a 

3 viewer of said computer. 

1 23. The computer program recorded on a machine-readable medium of claim 21 , wherein 

2 displaying said video message is performed by invoking the operation of a video display module. 

1 24. The computer program recorded on a machine-readable medium of claim 15, further 

2 performing the step of: 

3 in response to an indication that said receiver computer is not online, optionally sending 

4 said video message from said communication module of said sender computer to a host 

5 computer, for storage and later transmission to said communication module of said 

6 receiver computer, independent of the operation of said server computer. 

1 25. The computer program recorded on a machine-readable medium of claim 1 5, wherein 

2 said video message comprises video information. 

1 26. The computer program recorded on a machine-readable medium of claim 25, wherein 

2 said video message further comprises information selected from audio information, a thumbnail 

3 image, a text identifying a subject, a message body text, a document attachment, a slide and 

4 meta-data. 

1 27. A computer program recorded on a machine-readable medium, the computer program 

2 adapted to operate a server computer and to provide server functionality including database 

3 connectivity and communication protocols, the computer program when operating on a server 

4 computer performing the steps of: 

5 determining the online status of said receiver computer; and 

6 providing an indication of said status to said sender computer, such that in response to an 

7 indication that said receiver computer is online, streaming said video message from said 
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8 computer program of said sender computer to said computer program of said receiver computer 

9 is accomplished independent of the operation of said server computer. 

1 28. The computer program recorded on a machine-readable medium of claim 27, the 

2 computer program performing the further step of: 

3 accepting a request from a sender computer about the status of a receiver computer, said 

4 receiver computer being the intended recipient of a video message. 

1 29. The computer program recorded on a machine-readable medium of claim 27, the 

2 computer program performing the further step of: 

3 providing communication protocols for exchanging information between interconnected 

4 computers. 

1 30. The computer program recorded on a machine-readable medium of claim 27, the 

2 computer program performing the further step of: 

3 providing a database in which to store information, including information about sender 

4 and receiver computers authorized to communicate with said server computer. 
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